Information processing apparatus, imaging control method, program, digital microscope system, display control apparatus, display control method, and program including detection of a failure requiring reimaging

ABSTRACT

Provided is an information processing apparatus including a detection unit configured to detect a failure requiring reimaging relating to an image captured using a digital microscope by evaluating the image, and a generation unit configured to, if the failure was detected by the detection unit, generate setting information for setting an imaging condition for during reimaging.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit under 35 U.S.C. § 120 as acontinuation application of U.S. application Ser. No. 14/369,227, filedon Jun. 27, 2014, now U.S. Pat. No. 10,509,218, which claims the benefitunder 35 U.S.C. § 371 as a U.S. National Stage Entry of InternationalApplication No. PCT/JP2012/081801, filed in the Japanese Patent Officeas a Receiving Office on Dec. 7, 2012, which claims priority to JapanesePatent Application Number JP 2012-003214, filed in the Japanese PatentOffice on Jan. 11, 2012, each of which is hereby incorporated byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an imaging control method, a program, a digital microscope system, adisplay control apparatus, a display control method, and a program.

BACKGROUND ART

In the past, a digital microscope system has been utilized that capturesa digital image of a sample using a microscope. For example, in thefield of pathology diagnosis, by utilizing a digital microscope systemto capture a digital image of a biological sample, a pathology diagnosisbased on the biological sample can be carried out anywhere withoutretaining information about the biological sample for a long period ortransporting the biological sample itself. Patent Literature 1 disclosesa technology for generating high-resolution composite image data bycapturing partial images of a biological sample and combining thosepartial images.

CITATION LIST Patent Literature

Patent Literature 1: JP 2010-230495A

SUMMARY OF INVENTION Technical Problem

As illustrated in Patent Literature 1, in common digital microscopesystems, one sample image is generated from many partial images capturedusing the microscope. During the processes from image capture untilgeneration, failures often occur. For example, when automaticallycapturing many samples consecutively, it is rare for all of the imagecapturing of the sample to be successful. Despite this, in previoustechnologies relating to digital microscope systems, for example, whenan image has been obtained that will cause problems in utilizing asample image for pathology diagnosis or the like, namely, when there isa failure requiring reimaging, it takes a lot of time and effort fromdiscovery of that failure until reimaging. For example, that failuremight only be found by confirming that image on the screen.Consequently, a search is made for the sample to be reimaged, the sampleis reinserted, the imaging conditions are reset in order to avoidanother failure, and reimaging is carried out. Thus, reimaging takes alot of time and effort.

Therefore, it is desirable to provide a way to enable reimaging to beperformed more quickly and with less effort.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing apparatus including a detection unit configured to detect afailure requiring reimaging relating to an image captured using adigital microscope by evaluating the image, and a generation unitconfigured to, if the failure was detected by the detection unit,generate setting information for setting an imaging condition for duringreimaging.

According to another embodiment of the present disclosure, there isprovided an imaging control method including detecting a failurerequiring reimaging relating to an image captured using a digitalmicroscope by evaluating the image, and if the failure was detected bythe detection unit, generating setting information for setting animaging condition for during reimaging.

According to another embodiment of the present disclosure, there isprovided a program that causes a computer to function as a detectionunit configured to detect a failure requiring reimaging relating to animage captured using a digital microscope by evaluating the image, and ageneration unit configured to, if the failure was detected by thedetection unit, generate setting information for setting an imagingcondition for during reimaging.

According to another embodiment of the present disclosure, there isprovided a digital microscope system including a digital microscope, andan information processing apparatus including a detection unitconfigured to detect a failure requiring reimaging relating to an imagecaptured using the digital microscope by evaluating the image, and ageneration unit configured to, if the failure is detected by thedetection unit, generate setting information for setting an imagingcondition for during reimaging.

According to another embodiment of the present disclosure, there isprovided a display control apparatus including an acquisition unitconfigured to acquire setting information for setting an imagingcondition for during reimaging that is generated when a failurerequiring reimaging relating to an image captured using a digitalmicroscope is generated by evaluating the image, and a display controlunit configured to display the setting information acquired by theacquisition unit on a display face.

Advantageous Effects of Invention

According to the information processing apparatus, imaging controlmethod, program, digital microscope system, display control apparatus,display control method, and program according the present disclosure,reimaging can be performed more quickly and with less effort.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram illustrating an example of a schematicconfiguration of a digital microscope system according to an embodimentof the present disclosure.

FIG. 2 is a block diagram illustrating an example of a configuration ofa scanner according to a first embodiment.

FIG. 3 is an explanatory diagram illustrating an example of imaging by adigital microscope.

FIG. 4 is an explanatory diagram illustrating an example of combiningimages.

FIG. 5A is an explanatory diagram illustrating an example of an imagingomission.

FIG. 5B is an explanatory diagram illustrating an example of detectionof an imaging omission.

FIG. 6 is an explanatory diagram illustrating an example of edges thatappear in a composite image during a combining failure.

FIG. 7 is a block diagram illustrating an example of a configuration ofa server according to a first embodiment.

FIG. 8 is a block diagram illustrating an example of a configuration ofa viewer according to a first embodiment.

FIG. 9 is a flowchart illustrating an example of a schematic flow ofimaging control processing according to a first embodiment.

FIG. 10 is a flowchart illustrating an example of a flow of individualimage evaluation processing relating to an imaging omission.

FIG. 11 is a flowchart illustrating an example of a flow of individualimage evaluation processing relating to a combining failure.

FIG. 12 is a block diagram illustrating an example of a configuration ofa scanner according to a second embodiment.

FIG. 13 is a block diagram illustrating an example of a configuration ofa server according to a second embodiment.

FIG. 14 is a flowchart illustrating an example of a schematic flow ofimaging control processing on a server side according to a secondembodiment.

FIG. 15 is a flowchart illustrating an example of a flow of compositeimage evaluation processing relating to an imaging omission.

FIG. 16 is a flowchart illustrating an example of a flow of compositeimage evaluation processing relating to a combining failure.

FIG. 17 is a flowchart illustrating an example of a schematic flow ofimaging control processing on a scanner side according to a secondembodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the drawings, elements that have substantiallythe same function and structure are denoted with the same referencesigns, and repeated explanation is omitted.

It is noted that the description will be made in the following order.

-   1. Schematic configuration of a digital microscope system-   2. First embodiment-   2.1 Scanner configuration-   2.2 Server configuration-   2.3 Viewer configuration-   2.4 Processing flow-   3. Second embodiment-   3.1 Scanner configuration-   3.2 Server configuration-   3.3 Processing flow-   4. Summary

1. Schematic Configuration of a Digital Microscope System

First, a schematic configuration of a digital microscope system 1according to an embodiment of the present disclosure will now bedescribed with reference to FIG. 1. FIG. 1 is an explanatory diagramillustrating an example of a schematic configuration of the digitalmicroscope system according to an embodiment of the present disclosure.As illustrated in FIG. 1, the digital microscope system 1 includes ascanner 100, a server 200, and a viewer 300.

The scanner 100 is an information processing apparatus that has adigital microscope, or is connected to a digital microscope. The scanner100 captures an image of a sample using the digital microscope. Forexample, a prepared slide on which the sample is fixed is inserted inthe digital microscope. When the prepared slide has been inserted in thedigital microscope, the scanner 100 determines an imaging condition andsets the determined imaging condition in the digital microscope.Further, the scanner 100 captures an image of the sample on the preparedslide using the digital microscope. More specifically, the scanner 100determines an imaging area of the sample as an imaging condition, andsets the determined imaging area in the digital microscope. Then, thescanner 100 divides the imaging area into a plurality of individualareas, determines an imaging sequence of this plurality of individualareas, and sets the determined imaging sequence in the digitalmicroscope. Next, the scanner 100 makes the digital microscope capturean image of the sample at each individual area. Then, the scanner 100generates a composite image by combining the images of the plurality ofcaptured individual areas as partial images. Thus, one sample image,namely, a composite image, is generated by the scanner 100.

The server 200 is an information processing apparatus that manages asample image generated by the scanner 100. For example, the server 200associates and stores in a database the composite image generated by thescanner 100 with identification information about the sample.

The viewer 300 is an example of a display control apparatus. Forexample, the viewer 300 displays on a display face a composite image ofa sample generated by the scanner 100 or stored by the server 200.Further, for example, the viewer 300 provides a user interface forenabling the imaging conditions of the scanner 100 to be designatedbased on a user operation. Namely, the user can designate the imagingconditions of the scanner 100 via an operation made on the viewer 300.

In the embodiments according to the present disclosure, reimaging can becarried out in the digital microscope system 1 with less time andeffort. The specific subject matter of <2. First embodiment> and <3.Second embodiment> will now be described below.

2. First Embodiment

First, a first embodiment according to the present embodiment will bedescribed. According to the first embodiment of the present disclosure,a failure requiring reimaging is automatically detected by the scanner100, and reimaging is carried out based on that failure.

<2.1 Scanner Configuration>

An example of the configuration of a scanner 100-1 according to thefirst embodiment will now be described with reference to FIGS. 2 to 6.FIG. 2 is a block diagram illustrating an example of the configurationof the scanner 100-1 according to the first embodiment. As illustratedin FIG. 2, the scanner 100-1 includes an imaging control unit 110, adigital microscope 120, a combining unit 130, a failure detection unit140, a setting information generation unit 150, and a communication unit160.

(Imaging Control Unit 110)

The imaging control unit 110 controls imaging that uses the digitalmicroscope 120. For example, the imaging control unit 110 sets in thedigital microscope 120 imaging conditions designated by the user or thatare automatically determined, and makes the digital microscope 120capture an image of the sample based on those imaging conditions. Forexample, as imaging conditions for the entire sample, the imagingcontrol unit 110 sets, an imaging area, an imaging sequence,illumination brightness, a white balance coefficient and the like.Further, as an imaging condition for each individual area, for example,the imaging control unit 110 sets a focus position.

Further, when the setting information generation unit 150 has generatedthe setting information for setting the imaging conditions for duringreimaging, the imaging control unit 110 resets at least some of theabove-described imaging conditions in the digital microscope 120. Then,the imaging control unit 110 makes the digital microscope 120 recapturean image of the sample based on the reset imaging conditions.

The imaging control unit 110 outputs information about the set or resetimaging conditions (hereinafter referred to as “imaging conditionsinformation”) to the communication unit 160.

(Digital Microscope 120)

The digital microscope 120 captures an image based on the imagingconditions set by the imaging control unit 110. For example, the digitalmicroscope 120 captures images showing each individual area in the setimaging area (hereinafter, referred to as “individual images”). Thispoint will be described in more detail below with reference to FIG. 3.

FIG. 3 is an explanatory diagram illustrating an example of imaging bythe digital microscope 120. As illustrated in FIG. 3, to capture animage of a biological sample 10, an imaging area 20 that includes thewhole of the biological sample 10 is set. This imaging area 20 isdivided into a plurality of individual areas 21. The imaging sequence ofthe individual areas 21 can be set in an arbitrary sequence such as, forexample, in a spiral-shaped sequence from the center toward the externalside of the biological sample 10, or in a zigzag sequence thatalternately moves through each column from top to bottom or from bottomto top. Based on the thus-set imaging sequence, the digital microscope120 captures an individual image showing each of the individual areas21. Here, the digital microscope 120 captures an expanded area 23 thatincludes an individual area 21 as an individual image so that adjacentindividual images can be subsequently combined.

The digital microscope 120 outputs the captured individual image to thecombining unit 130 and the failure detection unit 140.

It is noted that the digital microscope 120 includes, for example, anoptical system, an image sensor, an image processing circuit, and adrive circuit. The optical system can include an objective lens, forexample. The image sensor is, for example, a CMOS (complementary metaloxide semiconductor) image sensor or a CCD (charge coupled device) imagesensor. The image processing circuit performs development processing,such as demosaicing processing and white balance correction. The drivecircuit moves the relative positions of the objective lens and thesample each time an individual image is captured based on the setimaging area and imaging sequence.

(Combining Unit 130)

The combining unit 130 generates a composite image by combiningindividual images captured by the digital microscope 120. This pointwill be described below in more detail with reference to FIG. 4.

FIG. 4 is an explanatory diagram illustrating an example of combiningimages. FIG. 4 shows individual images 30 a and 30 b captured atadjacent individual areas 21. As already described with reference toFIG. 3, the individual images 30 are images of an area 23 that includean individual area 21. Therefore, the individual images 30 have aportion 31 corresponding to the individual areas 21, and a portion 33(hereinafter referred to as “overlapping portion”) where the imagesoverlap. The combining unit 130 combines the individual image 30 a andthe individual image 30 b by appropriately superimposing an overlappingportion 33 b on an overlapping portion 33 a. Here, the combining unit130 superimposes the overlapping portion 33 b on the overlapping portion33 a using a luminance value of the overlapping portion 33 a; and aluminance value of the overlapping portion 33 b. As an example, forexample, the combining unit 130 superimposes the overlapping portion 33b on the overlapping portion 33 a, calculates a difference in theluminance value between each pixel of the overlapping portion 33 a andthe corresponding pixel of the overlapping portion 33 b, and sums theabsolute values of these differences across the plurality of pixels. Thecombining unit 130 calculates such a total value of the absolute valuesof the difference in luminance values for each position while changingthe position of the overlapping portion 33 b. Further, the combiningunit 130 determines the position of the overlapping portion 33 b wherethe minimum total value is calculated as the optimum position for theoverlapping portion 33 b during combining. The combining unit 130sequentially combines the individual images of adjacent individual areas21 in this way to ultimately generate one composite image of the entiresample.

The combining unit 130 outputs the generated composite image to thecommunication unit 160. Further, the combining unit 130 outputs theabove-described minimum total value regarding the combining of therespective individual images to the failure detection unit 140. It isnoted that the combining unit 130 may also output the generatedcomposite image to the failure detection unit 140.

(Failure Detection Unit 140)

The failure detection unit 140 detects a failure requiring reimagingrelating to an image captured using the digital microscope 120 byevaluating that image. The expression “evaluating an image” includesevaluating each of the individual images and evaluating a compositeimage formed from the individual images. Failure detection by thefailure detection unit 140 will be described below using first to fifthexamples of specific failure detection. It is noted that in the firstand second examples of failure detection, the failure detection unit 140detects a failure relating to a composite image generated by combining aplurality of individual images as the above-described failure.

In the first example, the failure detection unit 140 detects an imagingomission of an area that should be included in a composite image as afailure relating to a composite image. This point will be described inmore detail with reference to FIGS. 5A and 5B.

FIG. 5A is an explanatory diagram illustrating an example of an imagingomission. Similar to FIG. 3A, FIG. 5A illustrates a biological sample10, an imaging area 20, a plurality of individual areas 21, and an area23 that is an imaging area of an individual image and includes anindividual area 21. Here, although the imaging area 20 includes theentire dark-colored portion 11 of the biological sample 10, the imagingarea 20 does not include the light-colored portion 13 of the biologicalsample 10. Namely, a part of the sample 10 is not captured even thoughit is an area that should be included in the composite image. Forexample, when determining an imaging area by detecting the edges of abiological sample using an image of the whole of the biological sample,this light-colored portion 13 may thus be omitted from the imaging area.Thus, if there is an imaging omission, it can become difficult tocorrectly utilize the sample (e.g., in pathology diagnosis).

Accordingly, for example, the failure detection unit 140 detects animaging omission of an area that should be included in a composite imageby evaluating a direction in which texture is present in the image. Morespecifically, for example, the failure detection unit 140 evaluates adirection in which texture is present in an individual image of an areapositioned at a peripheral portion of the imaging area 20, namely, in anindividual image forming the peripheral portion of the composite image.This point will now be described in more detail with reference to FIG.5B.

FIG. 5B is an explanatory diagram illustrating an example of detectionof an imaging omission. FIG. 5B illustrates a single individual image 30that forms a peripheral portion 35 of a composite image. Since theindividual image 30 is an image of the area 23 that includes anindividual area 21, the individual image 30 includes a protrudingportion 37 that protrudes from the composite image. In other words, thisprotruding portion 37 shows an area on the external side of the imagingarea 20. Consequently, if some type of texture is present in theprotruding portion 37 of the individual image 30, it can be said thatthere is a sample that should also be captured in the direction of theprotruding portion 37 that is omitted from the imaging area 20.Therefore, the failure detection unit 140 can detect whether there is animaging omission by determining whether texture is present in thatprotruding portion 37 of the individual image 30 as an evaluation of thedirection in which texture is present. It is noted that the failuredetection unit 140 can detect whether texture is present in theprotruding portion 37 by, for example, detecting an edge in theprotruding portion 37 with an edge detection filter, calculating theamount of the detected edge, and determining whether that edge amountexceeds a predetermined threshold.

Further, the failure detection unit 140 may detect an imaging omissionusing the composite image rather than an individual image. In this case,the failure detection unit 140 detects the above-described imagingomission by evaluating a direction in which texture is present in anindividual image included in the composite image. Since the protrudingportion 37 of the individual image 30 is not included in the compositeimage, the failure detection unit 140 determines whether texture ispresent in the peripheral portion 35 instead of the protruding portion37 of the individual image 30, for example. If some kind of texture ispresent in the peripheral portion 35, it can be presumed that there is ahigh likelihood that there is also a sample that should be captured onthe external side of the imaging area 20 in the direction correspondingto the peripheral portion 35. Therefore, the failure detection unit 140can detect whether there is an imaging omission by determining whethertexture is present in the peripheral portion 35 as an evaluation of adirection in which texture is present. It is noted that the failuredetection unit 140 may determine the edge direction of the peripheralportion 35 instead of, or in combination with, determining whethertexture is present in the peripheral portion 35, for example. Namely,since it can be presumed that if an edge in the external direction ofthe composite image is present (in FIG. 5B, an edge in the verticaldirection), an edge is also extending in the protruding portion 37, itcan be presumed that there is a high likelihood that there is also asample that should be captured on the external side of the imaging area20 in the direction corresponding to the peripheral portion 35.

Thus, the failure detection unit 140 detects an imaging omission of anarea that should be included in the composite image by evaluating adirection in which texture is present in an individual image, forexample.

In a second example, the failure detection unit 140 detects a combiningfailure when generating the above-described composite image as a failurerelating to a composite image. As described with reference to FIG. 4,although a composite image is generated by combining captured individualimages, there are cases in which the individual images are notappropriately combined. Consequently, an unnatural shift can occuracross the whole of the composite image or in a part of the compositeimage, which can prevent the sample from being correctly confirmed.

Accordingly, as an example, the failure detection unit 140 detects theabove-described combining failure by evaluating the luminance value ofan overlapping portion when the composite image is generated. As alreadydescribed with reference to FIG. 4, when combining an individual image30 a and an individual image 30 b, the combining unit 130 calculates adifference in luminance values between each pixel of the overlappingportion 33 a and the corresponding pixel of the overlapping portion 33b, and sums the absolute values of these differences across theplurality of pixels. Further, the combining unit 130 determines theposition of the overlapping portion 33 b where the minimum total valueis calculated as the optimum position for the overlapping portion 33 bduring combining. Consequently, the above-described minimum total valuecan be said to be a value ultimately indicating the appropriatenesslevel of the combining. Namely, if the minimum total value is small, thecombining can be said to be successful, while if the minimum total valueis large, the combining can be said to be a failure. Therefore, thefailure detection unit 140 can detect a combining failure by determiningwhether the minimum total value exceeds a predetermined threshold. It isnoted that the failure detection unit 140 acquires the minimum totalvalue from the combining unit 130.

Further, as another example, the failure detection unit 140 may alsodetect the above-described combining failure by evaluating an edgeappearing in a predetermined direction in the composite image. Thispoint will be described in more detail below with reference to FIG. 6.

FIG. 6 is an explanatory diagram illustrating an example of edges thatappear in a composite image during combining failure. In FIG. 6, acomposite image 40 is illustrated. If a failure occurs in combining whengenerating the composite image 40, since the individual images areoffset from each other at the combining boundary, an edge appears at thecombining boundary. For example, an edge 41 a in the horizontaldirection and an edge 41 b in the vertical direction appear.Consequently, for example, the failure detection unit 140 detects edgesin predetermined directions (e.g., the horizontal direction and thevertical direction) that appear in the composite image, and evaluatesthe length or the strength of the detected edges. For example, if thelength of a detected edge is greater than a predetermined ratio of thelength of one side of an individual area 21, that edge can be presumedto be an edge generated as a result of combining failure. Further, if adetected edge has a strength that is greater than what could be expectedfor a particular edge of the sample, then that edge can also be presumedto be an edge generated as a result of combining failure. In such acase, the failure detection unit 140 detects the above-describedcombining failure.

Thus, the failure detection unit 140 detects a combining failure byevaluating, for example, the luminance value of an overlapping portionor an edge that appears in a predetermined direction in the compositeimage.

In a third example, the failure detection unit 140 detects a flaw in thefocus position as a failure requiring reimaging that is related to anindividual image or a composite image. When the digital microscope 120is capturing an individual image, the imaging control unit 110 sets afocus position in the digital microscope 120. However, if the focusposition setting is incorrect, a blurry image may be captured.Consequently, it can be difficult to correctly confirm the sample.

Accordingly, as an example, the failure detection unit 140 detects aflaw in the focus position by evaluating the contrast of the individualimage. Blurry images have a lower contrast. Therefore, the failuredetection unit 140 can detect a flaw in the focus position bycalculating the contrast of an individual image, and determining whetherthe calculated contrast exceeds a predetermined threshold. It is notedthat any method may be used to as the contrast calculation method. As anexample, the failure detection unit 140 calculates the difference inluminance values among each of four adjacent pixels, and sums theabsolute values of those differences. In addition, the failure detectionunit 140 may also sum the total value calculated for each pixel for thewhole individual image. For example, the thus-calculated total value maybe used as the contrast of the individual image. In this way, thefailure detection unit 140 detects a flaw in the focus position.

In a fourth example, the failure detection unit 140 detects a flaw inthe white balance as a failure requiring reimaging that is related to anindividual image or a composite image. Although white balance correctionis performed in the digital microscope 120 during the developmentprocessing of the individual image, a mistaken white balance can causethe color of the individual image to be developed as a color differentto the actual sample. Consequently, this can make it difficult tocorrectly confirm the color of the sample.

Accordingly, as an example, the digital microscope 120 captures an imageof an area in which the sample is not present (i.e., nothing is shown).Then, the failure detection unit 140 compares color information aboutthat image with color information prepared in advance as a template. Ifthe difference between these is greater than a predetermined threshold,the failure detection unit 140 detects a flaw in the white balance. Inthis way, the failure detection unit 140 detects a flaw in the whitebalance.

In a fifth example, the failure detection unit 140 detects a flaw in thebrightness as a failure requiring reimaging that is related to anindividual image or a composite image. Although the illuminationbrightness is adjusted during imaging in the digital microscope 120, amistaken brightness can cause individual images that are too bright orare too dark to be captured. Consequently, this can make it difficult tocorrectly confirm the sample.

Accordingly, as an example, similar to a flaw in the white balance, thedigital microscope 120 captures an image of an area in which the sampleis not present (i.e., nothing is shown). Then, the failure detectionunit 140 compares luminance information about that image with luminanceinformation prepared in advance as a template. If the difference betweenthese is greater than a predetermined threshold, the failure detectionunit 140 detects a flaw in the brightness. In this way, the failuredetection unit 140 detects a flaw in the brightness.

In the above, failure detection by the failure detection unit 140 wasdescribed using the first to fifth examples of specific failuredetection. Based on such failure detection, a failure requiringreimaging can be automatically detected without making the user confirmthe image on a screen.

In the first embodiment, every time each of a plurality of individualimages is captured, the failure detection unit 140 evaluates each imagein order to detect a failure requiring reimaging. Based on suchevaluation, a failure can be detected at the point when the individualimages are captured, which enables reimaging to be immediately performedafter the failure has been detected. Namely, in order to carry outreimaging, it is not necessary to reinsert the sample into the digitalmicroscope 120 or change the settings for items that do not need to bereset. Therefore, in this case, the work required to perform reimagingis almost eliminated, and the time taken for reimaging is substantiallyreduced.

The failure detection unit 140 outputs information relating to thedetected failure to the setting information generation unit 150.

(Setting Information Generation Unit 150)

When a failure is detected by the failure detection unit 140, thesetting information generation unit 150 generates setting informationfor setting the imaging conditions for during reimaging. Generation ofthe setting information by the setting information generation unit 150will be described below using the above-described first to fifthexamples of failure detection.

First, the generation of setting information for when an imagingomission is detected will be described. When the above-described imagingomission is detected by the failure detection unit 140, for example, thesetting information generation unit 150 generates imaging areainformation for resetting the imaging area so as to include anadditional area to be newly captured. This imaging area information isone piece of setting information. More specifically, as described withreference to FIGS. 5A and 5B, when evaluating the directions in whichtexture is present in an individual image, the failure detection unit140 knows the individual area 21 (or individual image 30) and thedirection (e.g., left, right, up, down) that the imaging omission is in.Therefore, the setting information generation unit 150 acquires from thefailure detection unit 140 a combination of the individual area 21coordinates and imaging omission direction as information relating tothat imaging omission, and generates information including thatcombination as area information. Based on such imaging area information,the imaging control unit 110 can reset the imaging area so as to includethe imaging omission area. Consequently, the same imaging omission canbe avoided during reimaging.

It is noted that the imaging area information may include informationsimply instructing that the imaging area be expanded. In this case, theimaging control unit 110 changes a parameter for determining the imagingarea so that the imaging area is wider. As an example, the imagingcontrol unit 110 determines the imaging area by detecting the edges ofthe sample that appear in an image of the whole sample. In this case,the imaging control unit 110 decreases the threshold in the edgedetection. Consequently, since weaker edges are also detected, a widerimaging area is determined and reset in the digital microscope 120.Further, as another example, the imaging control unit 110 divides animage of the whole sample into blocks, calculates the variance in theluminance value of each block, and determines a block area that has avariance exceeding a threshold as the imaging area. In this case, theimaging control unit 110 decreases this threshold for comparing with thevariance. Consequently, since more block areas are determined as theimaging area, a wider imaging area is reset in the digital microscope120.

In addition, for example, if the imaging area is reset, the settinginformation generation unit 150 also generates imaging sequenceinformation for resetting the imaging sequence of the individual images.This imaging sequence information is one piece of setting information.This imaging sequence information includes, for example, informationinstructing that the imaging sequence be newly set. Generally, theindividual images of adjacent individual areas 21 are capturedconsecutively. By thus consecutively capturing adjacent individualimages and combining them in sequence, the likelihood of a combiningfailure is decreased. On the other hand, in reimaging, if the individualimages are first captured in the original imaging sequence, and anindividual image of an expanded area is additionally captured, theindividual image of the expanded area is not captured in sequence withthe individual images of the original imaging area. Consequently, thelikelihood of a combining failure is increased. Therefore, whenresetting the imaging area, a combining failure can be avoided duringreimaging by additionally generating imaging sequence information andresetting the imaging sequence as well.

Second, the generation of setting information when a combining failureis detected during generation of the composite image will be described.When the above-described combining failure is detected by the failuredetection unit 140, the setting information generation unit 150generates imaging sequence information for resetting the imagingsequence of the individual images. This imaging sequence information isinformation that instructs an imaging sequence to be set that isdifferent from the imaging sequence when the combining failure wasdetected. This expression “different imaging sequence” may be an imagingsequence in which the start position of imaging is different, or animaging sequence in which the scan pattern (a spiral-shaped sequence, azigzag sequence etc.) is different. The occurrence of a combiningfailure is largely dependent on what type of imaging sequence is set. Asdescribed with reference to FIG. 4, the combining of the individualimages 30 is performed using a luminance value of an overlapping portion33. For example, when imaging from an individual image 30 in which thereis hardly any change in luminance among the pixels in the overlappingportion 33, a deviation occurs in combining, so that the likelihood of acombining failure in the imaging area increases. Thus, the occurrence ofa combining failure depends on the imaging sequence. Therefore, if acombining failure has occurred, by performing reimaging in a differentimaging sequence, the likelihood that a combining failure will occur isdecreased. It is noted that the imaging sequence information may includeinformation that indicates the imaging sequence per se when thecombining failure was detected.

Third, the generation of setting information when a flaw in the focusposition is detected will be described. When a flaw in the focusposition is detected, the setting information generation unit 150generates focus position information for resetting the focus position.This focus position information is one piece of setting information.This focus position information is, for example, information includingthe coordinates and the set focus position of an individual areacorresponding to the individual image for which a flow in the focusposition was detected. Based on such focus position information, theimaging control unit 110 can change the focus position especially for anindividual area having a flaw in its focus position. Consequently, thesame focus position flaw can be avoided during reimaging.

It is noted that, when determining the focus position at each individualarea in the imaging area, the imaging control unit 110 determines thefocus position for another individual area by, for example,independently measuring the focus position at several individual areas,and performing interpolation using the measured focus positions. In thiscase, based on this focus position, the imaging control unit 110 maychange the individual areas where the focus positions are independentlymeasured, or may change the interpolation method. In this case, thefocus position information may include information simply instructingthat the focus position be changed.

Fourth, the generation of setting information when a flaw in the whitebalance is detected will be described. When a flaw in the white balanceis detected, the setting information generation unit 150 generates whitebalance information for resetting the white balance. This white balanceinformation is one piece of setting information. This white balanceinformation includes, for example, color information about an image ofan area in which the sample is not present (i.e., color informationcompared with template color information) for when a flaw in whitebalance was detected. Based on such white balance information, theimaging control unit 110 can adjust the white balance (e.g., RGBcoefficient in white balance correction) to an appropriate value.Consequently, the same white balance flaw can be avoided duringreimaging. It is noted that the setting information generation unit 150may also determine how the RGB coefficient should be changed (e.g.,multiply R by a factor of 0.8, etc.) and generate white balanceinformation including such information from the color information aboutan image in which the sample is not present and the color informationprepared in advance as a template.

Fifth, the generation of setting information when a flaw in thebrightness is detected will be described. When a flaw in the brightnessis detected, the setting information generation unit 150 generatesbrightness information for resetting the illumination brightness. Thisbrightness information is one piece of setting information. Thisbrightness information includes, for example, luminance informationabout an image of an area in which the sample is not present (i.e.,luminance information compared with template luminance information) forwhen a flaw in brightness was detected. Based on such brightnessinformation, the imaging control unit 110 can adjust the illuminationbrightness to an appropriate value. Consequently, the same brightnessflaw can be avoided during reimaging. It is noted that the settinginformation generation unit 150 may also determine how the illuminationbrightness should be changed (e.g., multiply the brightness R by afactor of 1.2, etc.) and generate brightness information including suchinformation from the brightness information about an image in which thesample is not present and the brightness information prepared in advanceas a template.

In the above, the generation of setting information by the settinginformation generation unit 150 was described using the first to fifthexamples of specific failure detection. Based on such settinginformation, the scanner 100 can automatically set the imagingconditions for reimaging without the user having to individuallyreconsider the imaging conditions for avoiding another failure.

The setting information generation unit 150 outputs the generatedsetting information to the imaging control unit 110 and thecommunication unit 160.

(Imaging Control Unit 110 (Cont . . . ))

Thus, a failure requiring reimaging is detected by the combining unit130, and setting information based on this failure is generated by thesetting information generation unit 150. On the other hand, for example,the imaging control unit 110 determines whether to make the digitalmicroscope 120 recapture each of the individual images or to recapturethe entirety of the plurality of individual images based on the type ofthe above-described failure that is detected by the failure detectionunit 140. Namely, it is determined whether to perform partial reimagingor entire reimaging. For example, the imaging control unit 110 performsa determination as illustrated below.

TABLE 1 Failure Type Partial Reimaging Entire Reimaging Imaging omission∘ ∘ (time shortening > (failure avoidance > failure avoidance) timeshortening) Combining failure x ∘ Flaw in focus position ∘ ∘ (timeshortening > (failure avoidance > failure avoidance) time shortening)Flaw in white balance x ∘ Flaw in brightness x ∘

Namely, if the type of failure is an imaging omission or a flaw in thefocus position, for example, the imaging control unit 110 may determineto perform partial reimaging if giving priority to shortening the timetaken for reimaging. If giving priority to avoiding a combining failure,the imaging control unit 110 may determine to perform entire reimaging.Whether priority is given to shortening the time taken for reimaging orto avoiding a combining failure is set in the imaging control unit 110in advance.

Further, if the type of failure is a combining failure, a flaw in thewhite balance, or a flaw in the brightness, for example, the imagingcontrol unit 110 determines that entire reimaging is to be performed.Since a combining failure often affects the whole composite image ratherthan just a part of the composite image, it is desirable to performentire reimaging. In addition, since white balance and brightness arenormally set to the same setting for all of the plurality of individualimages rather than differently for each individual image, white balanceand brightness similarly effect the plurality of individual images.Therefore, it is desirable to perform entire reimaging.

Thus, by determining the reimaging range based on the type of failure,reimaging can be performed for the necessary range. Namely, reimagingcan be performed more efficiently while avoiding another failure in thereimaging.

(Communication Unit 160)

The communication unit 160 communicates with a server 200-1 and theviewer 300. For example, the communication unit 160 transmits thecomposite image from the combining unit 130 to the server 200-1 and theviewer 300. Further, the communication unit 160 transmits imagingconditions information from the imaging control unit 110 to the server200-1 and the viewer 300. In addition, the communication unit 160transmits setting information from the setting information generationunit 150 to the viewer 300.

Further, if the imaging conditions are to be determined or changed bythe server 200-1 or the viewer 300, the communication unit 160 may alsoreceive the imaging conditions information from the server 200-1 or theviewer 300. Similarly, if the setting information is to be generated orchanged by the server 200-1 or the viewer 300, the communication unit160 may also receive the setting information from the server 200-1 orthe viewer 300. The communication unit 160 can output these receivedpieces of information to the imaging control unit 110.

<2.2 Server Configuration>

An example of the configuration of the server 200-1 according to thefirst embodiment will now be described with reference to FIG. 7. FIG. 7is a block diagram illustrating an example of the configuration of theserver 200-1 according to the first embodiment. As illustrated in FIG.7, the server 200-1 includes a communication unit 210, a storage unit220, and a control unit 230.

(Communication Unit 210)

The communication unit 210 communicates with a scanner 100-1 and theviewer 300. For example, the communication unit 210 receives a compositeimage and imaging conditions information from the scanner 100-1.Further, the communication unit 210 transmits a composite image andimaging conditions information stored in the storage unit 220 to theviewer 300.

(Storage Unit 220)

The storage unit 220 stores a composite image and imaging conditionsinformation managed by the server 200-1. For example, the storage unit220 may also be configured as a database that associates identificationinformation about each sample, and stores a composite image of thatsample and imaging conditions information about when that compositeimage was generated.

(Control Unit 230)

The control unit 230 controls the whole server 200-1. For example, whena composite image and imaging conditions information are received by thecommunication unit 210 from the scanner 100-1, the control unit 230stores the received composite image and imaging conditions informationin the storage unit 220. Further, the control unit 230 may also transmitthe composite image to be presented to the user and the imagingconditions information either automatically or based on an instructionfrom the user.

<2.3 Viewer Configuration>

An example of the configuration of the viewer 300 according to the firstembodiment will now be described with reference to FIG. 8. FIG. 8 is ablock diagram illustrating an example of the configuration of the viewer300 according to the first embodiment. As illustrated in FIG. 8, theviewer 300 includes a communication unit 310, an input unit 320, acontrol unit 330, and a display unit 340.

(Communication Unit 310)

The communication unit 310 communicates with a scanner 100-1 and theserver 200-1. For example, the communication unit 310 receives acomposite image to be presented to the user from the scanner 100-1 orthe server 200-1. Along with the composite image, the communication unit310 can also receive imaging conditions information or settinginformation associated with that composite image.

Further, if the imaging conditions have been designated or the settinginformation has been edited by the user via a below-described userinterface, the communication unit 310 transmits the imaging conditionsinformation indicating the designated imaging conditions or the editedsetting information to the scanner 100-1.

(Input Unit 320)

The input unit 320 detects a user operation. This input unit 320 mayinclude one or more input devices, such as a touch panel, a keyboard,buttons, a pointing device and the like.

(Control Unit 330)

The control unit 330 controls the whole viewer 300. The control unit 330includes, for example, an acquisition unit 331 and a display controlunit 333.

(Acquisition Unit 331)

The acquisition unit 331 acquires a composite image to be presented tothe user and the imaging conditions information when that compositeimage was generated via the communication unit 310. The acquisition unit331 acquires the composite image and the imaging conditions informationbased on an instruction from the user, or each time a composite image isgenerated by the scanner 100-1, for example.

Further, the acquisition unit 331 acquires imaging conditionsinformation indicating the imaging conditions of the scanner 100-1 atthe current point via the communication unit 310. The acquisition unit331 acquires this imaging conditions information based on, for example,an instruction from the user.

In addition, the acquisition unit 331 may acquire setting informationvia the communication unit 310 under predetermined conditions. Forexample, if an instruction to present the setting information to theuser was issued by the user before imaging started, or if settinginformation that should be presented to the user has been generated, theacquisition unit 331 may acquire the setting information.

(Display Control Unit 333)

The display control unit 333 displays a composite image on a displayface of the display unit 340. For example, the display control unit 333displays a composite image on the above-described display face based onan instruction from the user or each time a composite image isgenerated. Further, if a failure requiring reimaging is found when theuser is viewing the composite image, the display control unit 333displays, for example, the imaging conditions information when thatcomposite image was generated on the display face of the display unit340 so that the user can reconsider the imaging conditions.

Further, to let the user freely designate the imaging conditions, thedisplay control unit 333 provides, for example, a user interface forenabling the imaging conditions of the scanner 100-1 to be designated bya user operation. For example, the display control unit 333 presents theimaging conditions of the scanner 100-1 at the current point to the userbased on an instruction from the user. Further, if the user designatesthe imaging conditions by a user operation on the input unit 320, thedisplay control unit 333 transmits imaging conditions informationindicating the designated imaging conditions to the scanner 100-1 viathe communication unit 310.

In addition, the display control unit 333 may also display acquiredsetting information on the display face of the display unit 340. In thepresent embodiment, a failure requiring reimaging is automaticallydetected and reimaging is also automatically carried out by the scanner100-1. However, if an instruction was issued by the user before imagingstarted to present setting information to the user, or if settinginformation that should be presented to the user has been generated,setting information for reimaging may also be displayed.

As an example, the display control unit 333 may present settinginformation to the user before reimaging is executed. Further, thedisplay control unit 333 may also enable the setting information to beedited by the user before reimaging is executed. Enabling the settinginformation to be edited in this way allows the user to edit the settinginformation and perform reimaging in a more appropriate manner if it isdetermined that the setting information is not suitable.

(Display Unit 340)

The display unit 340 is a display that has a display face. The displayunit 340 displays, for example, a composite image, imaging conditionsinformation, setting information or the like, on the display face underthe control of the control unit 330.

<2.4 Processing Flow>

Next, an example of the imaging control processing according to thefirst embodiment will be described with reference to FIGS. 9 to 11.

(Imaging Control Processing)

FIG. 9 is a flowchart illustrating an example of a schematic flow ofimaging control processing according to the first embodiment.

First, steps S401 to S405 are processes relating to entire imaging. Instep S401, the imaging control unit 110 in the scanner 100-1 determinesan imaging area, and sets the determined imaging area in the digitalmicroscope 120. Next, in step S403, the imaging control unit 110determines the imaging sequence of the individual images that show eachindividual area included in the set imaging area, and sets thedetermined imaging sequence in the digital microscope 120. Next, in stepS405, the imaging control unit 110 determines the other entire imagingconditions, such as the white balance and the illumination brightnessfor example, and sets those determined imaging conditions in the digitalmicroscope 120.

Next, steps S407 to S415 are processes that are repeated for eachindividual image.

In step S407, the imaging control unit 110 determines an individualimaging condition for capturing each individual image, such as the focusposition, for example, and sets the determined imaging condition in thedigital microscope 120. Next, in step S409, the digital microscope 120captures an individual image based on the set imaging sequence. Then, instep S411, the combining unit 130 combines the captured individual imagewith an adjacent individual image. Further, in step S500, the failuredetection unit 140 executes individual image evaluation processingrelating to the captured individual image.

In step S413, the failure detection unit 140 determines whether afailure requiring reimaging was detected in the individual imageevaluation processing. If a failure was detected, the processingproceeds to step S417. If a failure was not detected, the processingproceeds to step S415.

In step S415, the imaging control unit 110 determines whether imaging ofthe individual images has been completed for all of the individual areasincluded in the imaging area. If imaging has been completed, theprocessing finishes. If imaging has not been completed, the processingreturns to step S407.

In step S417, the setting information generation unit 150 generatessetting information for setting the imaging conditions for duringreimaging. Then, in step S419, the imaging control unit 110 determineswhether to perform partial reimaging or entire reimaging. If entirereimaging is desirable, the processing returns to step S401. If entirereimaging is not desirable, the processing returns to step S407. It isnoted that if the processing returns to step S401, in steps S401 to 407,the imaging control unit 110 sets the entire imaging conditions and theindividual imaging condition based on the generated setting information.Further, if the processing returns to step S407, in step S407, theimaging control unit 110 sets the individual imaging condition based onthe generated setting information.

(Individual Image Evaluation Processing 500)

Next, an example of the individual image evaluation processing 500 willbe described. The individual image evaluation processing 500 is executedfor each type of failure requiring reimaging. Namely, if five types offailure are detected, five individual image evaluation processes 500 areexecuted. Here, individual image evaluation processing 500 a relating toan imaging omission and individual image evaluation processing 500 brelating to a combining failure in particular will be described.

FIG. 10 is a flowchart illustrating an example of a flow of theindividual image evaluation processing 500 a relating to an imagingomission. In this individual image evaluation processing 500 a, adirection in which texture is present in an individual image isevaluated.

First, in step S510, the failure detection unit 140 determines whetheran individual image is an individual image of an area positioned at aperipheral portion of the imaging area 20 (i.e., an individual imageforming the peripheral portion of the composite image). If thisindividual image is an individual image of an area positioned at aperipheral portion of the imaging area 20, the processing proceeds tostep S520, and if not, the processing finishes.

In step S520, the failure detection unit 140 detects an edge of aprotruding portion of the individual image (i.e., a portion of theindividual image that protrudes from the composite image), andcalculates the amount of the detected edge.

Then, in step S530, the failure detection unit 140 determines whetherthe calculated edge amount exceeds a threshold. Namely, the failuredetection unit 140 determines whether texture is present in theprotruding portion of the individual image. If the edge amount exceedsthe threshold, the processing proceeds to step S540, and if not, theprocessing finishes.

In step S540, the failure detection unit 140 detects imaging omissionsin the area that should be included in the composite image as a failurerequiring reimaging. Then, the processing finishes.

Next, FIG. 11 is a flowchart illustrating an example of a flow of theindividual image evaluation processing 500 b relating to a combiningfailure. In this individual image evaluation processing 500 b, aluminance value of an overlapping portion when the composite image isgenerated is evaluated.

First, in step S550, the failure detection unit 140 calculates thedifference in luminance values between the overlapping portion ofindividual images that are adjacent to each other (i.e., the total valueof the absolute values of the difference in luminance values between thepixels corresponding to the overlapping portion).

Then, in step S560, the failure detection unit 140 determines whetherthe difference in luminance values of the above-described overlappingportion exceeds a predetermined threshold. If this difference exceedsthe predetermined threshold, the processing proceeds to step S570, andif not, the processing finishes.

In step S570, the failure detection unit 140 detects a combining failurewhen generating the composite image as a failure requiring reimaging.Then, the processing finishes.

In the above-described, a first embodiment was described. According tothe first embodiment, in the scanner 100, a failure requiring reimagingis automatically detected, and imaging conditions for during reimagingare automatically set based on setting information generated based onthe failure. Consequently, the user does not have to perform tasks, suchas confirming the image on the screen and resetting imaging conditionsand the like in order to avoid the failure from occurring again, as hasbeen carried out in the past. Therefore, reimaging can be carried outwith less effort and in less time. In addition, in the first embodiment,after a failure has been detected, reimaging can be immediately carriedout. Namely, in order to carry out reimaging, it is not necessary toreinsert the sample into the digital microscope 120 or change thesettings for items that do not need to be reset. Therefore, in thiscase, the work required to perform reimaging is almost eliminated, andthe time taken for reimaging is substantially reduced.

3. Second Embodiment

Next, a second embodiment of the present disclosure will be described.According to the second embodiment of the present disclosure, in aserver 200-2, a failure requiring reimaging is automatically detected.Further, based on the detection of this failure, the server 200-2prompts a scanner 100-2 to recapture an image of the sample. It is notedthat the configuration of the viewer 300 according to the secondembodiment may be the same as the configuration of the viewer 300according to the first embodiment.

<3.1 Scanner Configuration>

First, an example of the configuration of a scanner 100-2 according tothe second embodiment will be described with reference to FIG. 12. FIG.12 is a block diagram illustrating an example of the configuration ofthe scanner 100-2 according to the second embodiment. As illustrated inFIG. 12, the scanner 100-2 includes an imaging control unit 111, adigital microscope 121, a combining unit 131, and a communication unit160.

(Imaging Control Unit 111)

The imaging control unit 111 controls imaging that uses the digitalmicroscope 121. For example, the imaging control unit 110 sets in thedigital microscope 121 imaging conditions designated by the user or thatare automatically determined, and makes the digital microscope 121capture an image of the sample based on those imaging conditions.Similar to the imaging control unit 110 according to the firstembodiment, as imaging conditions of the whole sample, the imagingcontrol unit 111 sets, for example, an imaging area, an imagingsequence, illumination brightness, a white balance coefficient and thelike. Further, as an imaging condition for each individual area, forexample, the imaging control unit 111 sets a focus position.

Further, according to the present embodiment, when the settinginformation generated by the server 200-2 has been received by thecommunication unit 160, the imaging control unit 111 resets at leastsome of the above-described imaging conditions in the digital microscope121. Then, the imaging control unit 111 makes the digital microscope 121recapture an image of the sample based on the reset imaging conditions.

The imaging control unit 110 outputs imaging conditions informationindicating the set or reset imaging conditions to the communication unit160.

(Digital Microscope 121)

The digital microscope 121 captures an image based on the imagingconditions set by the imaging control unit 111. For example, similar tothe digital microscope 120 according to the first embodiment, thedigital microscope 121 captures individual images showing eachindividual area in the set imaging area. Further, the digital microscope121 outputs the captured individual images to the combining unit 131.

(Combining Unit 131)

Similar to the combining unit 130 according to the first embodiment, thecombining unit 131 generates a composite image by combining theindividual images captured by the digital microscope 121. Further, thecombining unit 131 outputs the generated composite image to thecommunication unit 160.

<3.2 Server Configuration>

Next, an example of the configuration of the server 200-2 according tothe second embodiment will be described with reference to FIG. 13. FIG.13 is a block diagram illustrating an example of the configuration ofthe server 200-2 according to the second embodiment. As illustrated inFIG. 13, the server 200-2 includes a communication unit 210, a storageunit 220, a control unit 230, a failure detection unit 240, and asetting information generation unit 250.

(Failure Detection Unit 240)

The failure detection unit 240 detects a failure requiring reimaging byevaluating a composite image (or a partial image thereof) received fromthe scanner 100-2 via the communication unit 210. The types of failuredetected by the failure detection unit 240 may be the same as the typesof failure that are detected by the failure detection unit 140 of thescanner 100-1 according to the first embodiment. Namely, the failuredetection unit 240 can detect the above-described imaging omission andcombining failure as failures relating to a composite image. Further, asother failures, the failure detection unit 240 can detect theabove-described flaw in focus position, flaw in white balance, and flawin brightness.

It is noted that when detecting an imaging omission, since a protrudingportion is not included in the composite image, the failure detectionunit 240 uses the peripheral portion 35 illustrated in FIG. 5B in theevaluation of the direction in which texture is present in the compositeimage or partial image. Further, when detecting a combining failure,since the two overlapping portions of the individual images that areadjacent to each other have already been lost, the failure detectionunit 240 evaluates the direction of edges included in the compositeimage as illustrated in FIG. 6.

The failure detection unit 240 outputs information relating to thedetected failure to the setting information generation unit 250.

(Setting Information Generation Unit 250)

When a failure is detected by the failure detection unit 240, thesetting information generation unit 250 generates setting informationfor setting the imaging conditions for during reimaging. Similar to thesetting information generation unit 150, the setting informationgeneration unit 250 generates various types of setting information whena failure requiring reimaging is detected, such as an imaging omission,a combining failure, a flaw in the focus position, a flaw in the whitebalance, and a flaw in the brightness.

When the setting information has been generated, the setting informationgeneration unit 250 transmits to the scanner 100-2 the settinginformation and a reimaging request prompting reimaging via thecommunication unit 210. The reimaging request may also be transmitted tothe scanner 100-2 after the reimaging request has been transmitted tothe viewer 300 and a reimaging instruction from the user input on theviewer 300.

<3.3 Processing Flow>

Next, an example of the imaging control processing according to thesecond embodiment will be described with reference to FIGS. 14 to 17.

(Imaging Control Processing on the Server 200-2 Side)

First, of the imaging control processing, an example of the processingperformed on the server 200-2 side will be described. FIG. 14 is aflowchart illustrating an example of a schematic flow of imaging controlprocessing on the server 200-2 side according to the second embodiment.

First, in step S601, the failure detection unit 240 of the server 200-2reads a composite image from the storage unit 220. For example, thefailure detection unit 240 selects and reads a composite image from thestorage unit 220 by automatically selecting an unevaluated compositeimage or based on an instruction from the user. Next, in step S700, thefailure detection unit 240 executes composite image evaluationprocessing. Then, in step S603, the failure detection unit 240determines whether a failure requiring reimaging has been detected. Ifsuch a failure has been detected, the processing proceeds to step S605,and if not, the processing finishes.

In step S605, the setting information generation unit 250 generatessetting information for setting the imaging conditions for duringreimaging. Next, in step S607, the setting information generation unit250 stores the generated setting information in the storage unit 220.Further, in step 609, the setting information generation unit 250transmits the setting information and a reimaging request to the scanner100-2 or the viewer 300 via the communication unit 210. Then, theprocessing finishes.

(Composite Image Evaluation Processing 700)

Next, an example of the composite image evaluation processing 700 willbe described. The composite image evaluation processing 700 is executedfor each type of failure requiring reimaging. Namely, if five types offailure are detected, five composite image evaluation processes 700 areexecuted. Here, composite image evaluation processing 700 a relating toan imaging omission and composite image evaluation processing 700 brelating to a combining failure in particular will be described.

FIG. 15 is a flowchart illustrating an example of a flow of thecomposite image evaluation processing 700 a relating to an imagingomission. In this composite image evaluation processing 700 a, adirection in which texture is present in a composite image is evaluated.

First, in step S710, the failure detection unit 240 detects an edge of aperipheral portion of a composite image, and calculates an amount of thedetected edge. The failure detection unit 240 calculates the edge amountfor each individual image (i.e., partial image) forming the compositeimage.

Then, in step S720, the failure detection unit 240 determines whetherthe calculated edge amount exceeds a threshold. Namely, the failuredetection unit 240 determines whether texture is present in a peripheralportion of the composite image. The failure detection unit 240 performsa determination for each individual image (i.e., partial image) formingthe composite image. If the edge amount exceeds the threshold, theprocessing proceeds to step S730, and if not, the processing finishes.

In step S730, the failure detection unit 240 detects imaging omissionsin the area that should be included in the composite image as a failurerequiring reimaging. Then, the processing finishes.

Next, FIG. 16 is a flowchart illustrating an example of a flow of thecomposite image evaluation processing 700 b relating to a combiningfailure. In this composite image evaluation processing 700 a, an edge ina predetermined direction is evaluated.

First, In step S740, the failure detection unit 240 detects an edge in apredetermined direction that is included in the composite image, andcalculates the length or strength of the detected edge.

Then, in step S750, the failure detection unit 240 determines if thereis an edge in the predetermined direction that is equal to or greaterthan a predetermined length or a predetermined strength. If there is anedge in the predetermined direction that is equal to or greater than thepredetermined length or the predetermined strength, the processingproceeds to step S760, and if not, the processing finishes.

In step S760, the failure detection unit 240 detects a combining failurewhen generating the composite image as a failure requiring reimaging.Then, the processing finishes.

(Imaging Control Processing on the Scanner 100-2 Side)

Next, of the imaging control processing, an example of the processingperformed on the scanner 100-2 side will be described. FIG. 17 is aflowchart illustrating an example of a schematic flow of imaging controlprocessing on the scanner 100-2 side according to the second embodiment.

First, steps S801 to S807 are processes relating to entire imaging. Instep S801, the communication unit 160 of the scanner 100-2 receivessetting information from the server 200-2. Next, in step S803, based onthe setting information, the imaging control unit 111 determines theimaging sequence of the individual images showing each of the individualareas included in the set imaging area, and sets the determined imagingsequence in the digital microscope 121. Further, in step S807, based onthe setting information, the imaging control unit 111 determines theother entire imaging conditions, such as the while balance and theillumination brightness for example, and sets the determined imagingconditions in the communication circuit 121.

Next, steps S809 to S815 are processes that are repeated for eachindividual image.

In step S809, the imaging control unit 111 determines an individualimaging condition for capturing each individual image, such as the focusposition, for example, and sets the determined imaging condition in thedigital microscope 121. Next, in step S811, the digital microscope 121captures an individual image based on the set imaging sequence. Then, instep S813, the combining unit 131 combines the captured individual imagewith an adjacent individual image.

In step S815, the imaging control unit 111 determines whether imaging ofthe individual images has been completed for all of the individual areasincluded in the imaging area. If imaging has been completed, theprocessing proceeds to step S817. If imaging has not been completed, theprocessing returns to step S809.

In step S817, the communication unit 160 transmits the composite imageto the server 200-2 and the viewer 300. Then, the processing finishes.

In the above-described, a second embodiment was described. According tothe second embodiment, in the scanner 200, a failure requiring reimagingis automatically detected, and the scanner 100-2 is prompted to executereimaging based on that failure. Consequently, the user does not have toperform tasks, such as confirming the image on the screen and resettingimaging conditions and the like in order to avoid the failure fromoccurring again, as has been carried out in the past. Therefore,reimaging can be carried out with less effort and in less time. Inaddition, in the second embodiment, failure detection and settinginformation generation can be performed by storing a composite image inthe server even without having failure detection and setting informationgeneration functions in each scanner. Further, failure detection andsetting information generation can be performed as necessary ex-postfacto while reducing the imaging time at a scanner.

Summary

In the above, two embodiments of the digital microscope system accordingto the present disclosure were described with reference to FIGS. 1 to17. According to these embodiments of the present disclosure, a failurerequiring reimaging is detected by evaluating an image captured using adigital microscope. Further, when such a failure is detected, settinginformation for setting imaging conditions for during reimaging isgenerated. Namely, a failure requiring reimaging is automaticallydetected, and imaging conditions for during reimaging are automaticallyset based on setting information generated based on the failure.Consequently, the user does not have to perform tasks, such asconfirming the image on the screen and resetting imaging conditions andthe like in order to avoid the failure from occurring again, as has beencarried out in the past. Therefore, reimaging can be carried out withless effort and in less time.

Further, a failure relating to a composite image is detected as afailure requiring reimaging. Based on the detection of such a failure,reimaging is automatically performed even if a failure peculiar to adigital microscope system occurs, such as an imaging omission and acombining failure. Consequently, the time and effort for reimaging in adigital microscope system can be reduced.

Still further, if an imaging omission is detected, imaging areainformation is generated as setting information. Based on this imagingarea information, an imaging area can be set so as to include the areaof the imaging omission. This enables the same imaging omission to beavoided during reimaging. In addition, imaging sequence information isgenerated as setting information with this imaging area information.Based on this imaging sequence information, a combining failure can beavoided during reimaging.

Moreover, if a combining failure is detected, imaging sequenceinformation is generated as setting information. Based on this imagingsequence information, the likelihood of a combining failure occurringcan be reduced by performing reimaging in a different imaging sequence.

Further, during reimaging, a determination is made whether to performeither partial reimaging or entire reimaging based on the type ofdetected failure. Thus, reimaging can be performed for the necessaryrange by determining the reimaging range based on the type of failure.Namely, reimaging can be performed more efficiently while avoidinganother failure in the reimaging.

Still further, failure detection and setting information generation maybe performed on the scanner side, for example. In this case, forexample, every time the plurality of individual images are captured,each image is evaluated in order to detect a failure requiringreimaging. Since such an evaluation enables a failure to be detectedafter image capturing, reimaging can be performed immediately.Consequently, since reimaging can be performed without rearranging theimaging target, such as a prepared slide, every time, the time andeffort taken for the reimaging can be reduced.

In addition, failure detection and setting information generation may beperformed on the server side, for example. In this case, for example,failure detection and setting information generation are performed usinga composite image that has accumulated in the server. Consequently,failure detection and setting information generation can be performed bystoring a composite image in this server even without having failuredetection and setting information generation functions in each scanner.Further, failure detection and setting information generation can beperformed as necessary ex-post facto while reducing the imaging time ata scanner.

The preferred embodiments of the present invention have been describedabove with reference to the accompanying drawings, whilst the presentinvention is not limited to the above examples, of course. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present invention.

For example, the digital microscope may be configured without a scanner,or may be a separate apparatus from the scanner. In this case, thedigital microscope is connected to the scanner. Further, two or moreapparatuses from among the scanner, the server, and the viewer may bethe same apparatus.

Further, the processing steps in the imaging control processingaccording to the present specification do not have to be executed in thetemporal order described in the flowcharts. For example, the processingsteps in the imaging control processing may be carried out in an orderdifferent to that described in the flowcharts, or may be carried out inparallel.

In addition, a computer program can be written that makes a CPU, ROM,RAM and the like included in a scanner, server, and viewer exhibit thesame functions as the respective parts in the above-described scanner,server, and viewer. Moreover, a storage medium in which such a computerprogram is stored is also provided.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

a detection unit configured to detect a failure requiring reimagingrelating to an image captured using a digital microscope by evaluatingthe image; and

a generation unit configured to, if the failure was detected by thedetection unit, generate setting information for setting an imagingcondition for during reimaging.

(2)

The information processing apparatus according to (1), wherein thedetection unit is configured to detect as the failure a failure relatingto a composite image generated by combining a plurality of images.

(3)

The information processing apparatus according to (2), wherein thefailure relating to the composite image includes an imaging omission ofan area that should be included in the composite image.

(4)

The information processing apparatus according to (3), wherein thedetection unit is configured to detect the imaging omission byevaluating a direction in which texture is present in the image.

(5)

The information processing apparatus according to any one of (2) to (4),wherein the generation unit is configured to, if the imaging omissionwas detected by the detection unit, generate imaging area informationfor resetting an imaging area so as to include an additional area to benewly captured.

(6)

The information processing apparatus according to (5), wherein thegeneration unit is configured to, if the imaging area is to be reset,further generate imaging sequence information for resetting an imagingsequence of the image.

(7)

The information processing apparatus according to (2), wherein thefailure relating to the composite image includes a combining failurewhen generating the composite image.

(8)

The information processing apparatus according to (7), wherein thedetection unit is configured to detect the combining failure byevaluating an edge in a predetermined direction included in thecomposite image.

(9)

The information processing apparatus according to (7), wherein thedetection unit is configured to detect the combining failure byevaluating a luminance value of an overlapping portion when thecomposite image is generated.

(10)

The information processing apparatus according to any one of (7) to (9),wherein the generation unit is configured to, if the combining failurewas detected by the detection unit, generate imaging sequenceinformation for resetting an imaging sequence of the image.

(11)

The information processing apparatus according to any one of (2) to(10), wherein the detection unit is configured to further detect as thefailure a flaw in focus position, white balance, or brightness.

(12)

The information processing apparatus according to any one of (2) to(11), further including:

an imaging control unit configured to determine whether to make thedigital microscope recapture each image or recapture an entirety of aplurality of images based on a type of the failure detected by thedetection unit.

(13)

The information processing apparatus according to any one of (2) to(12), wherein the detection unit is configured to evaluate each imagefor detection of the failure every time each of the plurality of imagesis captured.

(14)

The information processing apparatus according to any one of (2) to(12), wherein the detection unit is configured to evaluate the compositeimage or a partial image of the composite image after the compositeimage has been generated.

(15)

The information processing apparatus according to any one of (1) to(14), further including:

a display control unit configured to enable the setting informationgenerated by the generation unit to be presented to the user or editedby the user before reimaging is executed.

(16)

An imaging control method including:

detecting a failure requiring reimaging relating to an image capturedusing a digital microscope by evaluating the image; and

if the failure was detected by the detection unit, generating settinginformation for setting an imaging condition for during reimaging.

(17)

A program that causes a computer to function as:

a detection unit configured to detect a failure requiring reimagingrelating to an image captured using a digital microscope by evaluatingthe image; and

a generation unit configured to, if the failure was detected by thedetection unit, generate setting information for setting an imagingcondition for during reimaging.

(18)

A digital microscope system including:

a digital microscope; and

an information processing apparatus including

-   -   a detection unit configured to detect a failure requiring        reimaging relating to an image captured using the digital        microscope by evaluating the image, and    -   a generation unit configured to, if the failure is detected by        the detection unit, generate setting information for setting an        imaging condition for during reimaging.        (19)

A display control apparatus including:

an acquisition unit configured to acquire setting information forsetting an imaging condition for during reimaging that is generated whena failure requiring reimaging relating to an image captured using adigital microscope is generated by evaluating the image; and

a display control unit configured to display the setting informationacquired by the acquisition unit on a display face.

(20)

An information processing apparatus including:

a detection unit configured to detect a failure relating to a compositeimage generated by combining a plurality of images captured using adigital microscope by evaluating an image; and

a generation unit configured to, if the failure is detected by thedetection unit, generate setting information for setting an imagingcondition for during reimaging.

REFERENCE SIGNS LIST

-   1 digital microscope system-   10 biological sample-   20 imaging area-   21 individual area-   30 individual image-   33 overlapping portion-   35 peripheral portion-   37 protruding portion-   40 composite image-   100 scanner-   110, 111 imaging control unit-   120, 121 digital microscope-   130, 131 combining unit-   140 failure detection unit-   150 setting information generation unit-   160 communication unit-   200 server-   210 communication unit-   220 storage unit-   230 control unit-   240 failure detection unit-   250 setting information generation unit-   300 viewer-   310 communication unit-   320 input unit-   330 control unit-   331 acquisition unit-   333 display control unit-   340 display unit

The invention claimed is:
 1. An information processing apparatuscomprising: circuitry configured to: obtain a first image and a secondimage of a target object; detect a combining failure based on comparisonbetween luminance values of each pixel of a first overlapping portion ofthe first image and each corresponding pixel of a second overlappingportion of the second image; combine the first image and the secondimage based on the luminance values by determining an overlap positionof the first image and second image in a case that the combining failureis not detected; and output information related to the combining failurein a case that the combining failure is detected, wherein detecting acombining failure includes detecting an edge, in a predetermineddirection, in a composite image generated by combining a plurality ofimages, calculating a length of the edge in the predetermined direction,and determining a combining failure when the calculated length exceeds apredetermined length.
 2. The information processing apparatus accordingto claim 1, wherein the circuitry is configured to, in the case that thecombining failure is detected, generate imaging sequence information forresetting an imaging sequence of the image.
 3. The informationprocessing apparatus according to claim 1, wherein the circuitry isconfigured to further detect as the failure a flaw in focus position,white balance, or brightness.
 4. The information processing apparatusaccording to claim 1, wherein the circuitry is further configured todetermine whether to recapture each image or recapture an entirety of aplurality of images based on a type of the failure detected.
 5. Theinformation processing apparatus according to claim 1, wherein thecircuitry is configured to evaluate each image for detection of thefailure every time each of the plurality of images is captured.
 6. Theinformation processing apparatus according to claim 1, wherein thecircuitry is configured to evaluate the composite image or a partialimage of the composite image after the composite image has beengenerated.
 7. The information processing apparatus according to claim 1,wherein, in the case that the combining failure is detected, thecircuitry is configured to output setting information for setting animaging condition during reimaging.
 8. The information processingapparatus according to claim 7, wherein the circuitry is furtherconfigured to enable the setting information to be presented to the useror edited by the user before reimaging is executed.
 9. The informationprocessing apparatus according to claim 1, wherein the first image andthe second image are obtained by a digital microscope.
 10. An imagingcontrol method comprising: obtaining a first image and a second image ofa target object; detecting a combining failure based on comparisonbetween luminance values of each pixel of a first overlapping portion ofthe first image and each corresponding pixel of a second overlappingportion of the second image; combining the first image and the secondimage based on the luminance values by determining an overlap positionof the first image and second image in a case that the combining failureis not detected; and outputting information related to the combiningfailure in a case that the combining failure is detected, whereindetecting a combining failure includes detecting an edge, in apredetermined direction, in a composite image generated by combining aplurality of images, calculating a length of the edge in thepredetermined direction, and determining a combining failure when thecalculated length exceeds a predetermined length.
 11. Acomputer-readable storage device encoded with computer-executableinstructions that, when executed by a computer, cause the computer to:obtain a first image and a second image of a target object; detect acombining failure based on comparison between luminance values of eachpixel of a first overlapping portion of the first image and eachcorresponding pixel of a second overlapping portion of the second image;combine the first image and the second image based on the luminancevalues by determining an overlap position of the first image and secondimage in a case that the combining failure is not detected; and outputinformation related to the combining failure in a case that thecombining failure is detected, wherein detecting a combining failureincludes detecting an edge, in a predetermined direction, in a compositeimage generated by combining a plurality of images, calculating a lengthof the edge in the predetermined direction, and determining a combiningfailure when the calculated length exceeds a predetermined length.
 12. Adigital microscope system comprising: a digital microscope; and aninformation processing apparatus including circuitry configured to:obtain a first image and a second image of a target object; detect acombining failure based on comparison between luminance values of eachpixel of a first overlapping portion of the first image and eachcorresponding pixel of a second overlapping portion of the second image;combine the first image and the second image based on the luminancevalues by determining an overlap position of the first image and secondimage in a case that the combining failure is not detected; and outputinformation related to the combining failure in a case that thecombining failure is detected, wherein detecting a combining failureincludes detecting an edge, in a predetermined direction, in a compositeimage generated by combining a plurality of images, calculating a lengthof the edge in the predetermined direction, and determining a combiningfailure when the calculated length exceeds a predetermined length.